Automation of cleaning and ensembles for outliers detection in questionnaire data

This article is focused on the automatic detection of corrupted or inappropriate responses in questionnaire data using unsupervised outliers detection. The surveys are often used psychology research to collect self-report and their preprocessing takes a lot manual effort. Unlike with numerical where distance-based prevail, records questionnaires have be assessed from various perspectives that do not relate so much. We identify most frequent types errors questionnaires. For each them, we suggest different methods ranking usage normalized scores. Considering similarity between pairs outlier scores (some highly uncorrelated), propose an ensemble based union detected by methods. Our framework consists some well-known algorithms but also novel approaches addressing typical issues selected distance, entropy, probability. experimental section describes process assembling selecting parameters for final model detecting significant real-world HBSC dataset.